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Dependable  Software  Technology  Exchange 


Abstract:  On  March  18  and  19.  1993,  the  Dependable  Real-Time  Software 
project  hosted  a  Dependable  Software  Technology  Exchange.  The  exchange, 
sponsored  by  the  Air  Force  Space  and  Missile  Systems  Center  and  the  Office 
of  Naval  Research,  brought  together  researchers  and  system  developers, 
providing  an  opportunity  for  the  researchers  to  learn  the  needs  of  the 
developers  and  for  the  developers  to  learn  about  techniques  being  investigated 
by  the  researchers.  This  report  summarizes  what  transpired  at  the  meeting. 


1  Introduction 

Dependability  is  important  in  avionics,  vehicle  control,  logistics  systems,  and  command,  con¬ 
trol,  and  communications  systems,  to  name  but  a  few.  As  these  systems  increasingly  rely  on 
computers,  software  becomes  more  important  to  the  dependability  and  safety  of  their  users. 
Computer  hardware  has  become  more  reliable,  so  software  is  now  the  bottleneck  for  achieving 
dependability.  In  recent  years,  however,  a  number  of  new  approaches  have  emerged  for  build¬ 
ing  software  for  dependable  system  applications. 

One  way  to  facilitate  technology  transfer  between  researchers  and  practitioners  is  to  get  them 
talking  to  each  other.  With  this  mind,  on  March  18  and  19, 1993,  the  Software  Engineering  In¬ 
stitute's  Dependable  Real-Time  Software  project  hosted  the  first  in  what  is  hoped  to  be  a  se¬ 
ries  of  Dependable  Software  Technology  Exchanges.  This  initial  technology  exchange  was 
sponsored  by  the  Air  Force  Space  and  Missile  Systems  Center  and  the  Office  of  Naval  Re¬ 
search. 

The  purpose  of  the  exchange  was  to  bring  researchers  and  system  builders  together  to  pro¬ 
vide  an  opportunity  for  technology  transfer  in  both  directions.  System  builders  are  often  too 
busy  to  attend  conferences  or  to  write  papers;  researchers  need  exposure  to  “real  world”  con¬ 
cerns.  The  technology  exchange  provided  a  forum  where  each  side  could  educate  the  other. 

The  exchange  was  coordinated  by  a  steering  committee  consisting  of: 

•  Dr.  Charles  B.  Weinstock,  Software  Engineering  Institute 

•  Dr.  Fred  B.  Schneider,  Cornell  University 

•  Dr.  Yitzak  Levendel,  AT&T  Bell  Laboratories 

•  Dr.  Ravi  Iyer,  University  of  Illinois 

•  Dr.  Walter  L.  Heimerdinger,  Honeywell  Systems  and  Research  Center 

•  Mr.  Joseph  Chiara,  Air  Force  Space  and  Missile  Systems  Center 

•  Dr.  Gul  Agha,  University  of  Illinois 

The  focus  was  on  four  topic  areas  that  were  deemed  to  be  critical  for  achieving  dependability: 
formal  methods  and  verification,  requirements,  operating  system  support,  and  object-oriented 
methods  and  design.  Each  area  was  allocated  a  half  day  session  divided  into  three  parts.  The 
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session  started  with  a  one  hour  presentation  from  a  researcher  followed  with  a  one  hour  long 
presentation  from  an  experienced  practitioner.  After  a  short  break,  a  75  minute  panel  discus¬ 
sion  was  held.  Each  panel  consisted  of  the  two  speakers,  plus  four  additional  panelists.  Audi¬ 
ence  participation  was  encouraged  throughout  the  sessions.  The  technology  exchange  was 
videotaped. 

By  all  accounts,  the  first  Dependable  Software  Technology  Exchange  was  successful.  It  was 
well  attended — demand  almost  exceeded  capacity.  Of  perhaps  greater  importance  is  the  di¬ 
versity  of  backgrounds  represented  by  the  attendees.  The  over  70  attendees  represented  or¬ 
ganizations  including: 


Industry 

• 

Universal  Hi-Tech 

• 

AEG/Westinghouse 

Development 

Transportation  Systems 

• 

Weinberg  &  Associates 

• 

AlliedSignal 

Academia 

• 

AT&T  Bell  Laboratories 

• 

Brown  University 

• 

Booz-Allen  and  Hamilton 

• 

Carnegie  Mellon  University 

• 

Digital  Equipment 

• 

Cornell  University 

• 

Honeywell 

• 

McMaster  University 

• 

Hughes  Aircraft 

• 

University  of  Arizona 

• 

IBM  Federal  Systems 

• 

University  of  California 

• 

Lockheed  Missile  and  Space 

• 

University  of  Illinois 

• 

Martin  Marietta 

• 

University  of  Pittsburgh 

• 

MCC 

• 

University  of  Texas 

• 

Mind  Tools  Corporation 

• 

University  of  York 

• 

ObjecTime  Limited 

Government 

• 

Rockwell  International 

• 

AFOSR 

• 

Siemens 

• 

AF/SMC 

• 

SOHAR 

• 

NASA 

• 

SPARTA,  Inc. 

• 

NIST 

• 

System  Technology 

• 

NRL 

Development 

• 

NSWC 

• 

Tandem  Computers 

• 

ONR 

• 

Texas  Instruments 

• 

SDIO 

• 

Transarc  Corporation 

Other 

• 

Aerospace  Corporation 

We  clearly  achieved  our  goal  of  bringing  researchers  and  practitioners  together. 


At  the  end  of  the  technology  exchange,  attendees  completed  a  feedback  form  to  help  us  eval¬ 
uate  the  success  of  the  meeting.  Although  not  every  respondent  found  all  of  the  talks  useful, 
every  talk  was  useful  to  some  of  the  respondents.  Virtually  every  respondent  was  enthusiastic 
about  holding  further  technology  exchanges,  and  a  variety  of  topics  were  suggested. 
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The  next  sections  describe  the  four  sessions  in  detail,  including  the  speakers  and  panelists  as 
well  as  a  summary  of  the  discussion.  Following  those  sections  is  a  synopsis  of  the  comments 
from  participants.  A  complete  list  of  attendees  appears  as  an  appendix. 
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2  Formal  Methods  and  Verification 


The  morning  session  of  Thursday,  March  18,  1993,  focused  on  formal  methods  and  verifica¬ 
tion.  The  technology  lecture  was  given  by  Dr.  John  Rushby  from  SRI  International.  The  title  of 
his  lecture  was  “Formal  Methods  Technology:  What  Can  it  Do?  How  Best  to  Use  it?"  Dr.  David 
Lorge  Pamas  of  McMaster  University  gave  the  application  lecture.  The  Application  of  Mathe¬ 
matical  Methods  for  Documentation  and  Inspection  of  Critical  Software.”  In  addition  to  these 
speakers,  the  panel  consisted  of  Mr.  Ricky  Butler  of  NASA  Langley  Research  Center,  Mr.  John 
Dehn  of  IBM  Federal  Systems,  Dr.  Bill  McKeeman  of  Digital  Equipment,  and  Dr.  Fred 
Schneider  of  Cornell  University.  The  session  was  coordinated  by  Dr.  Schneider. 

2.1  Technology  Lecture:  Dr.  John  Rushby 

Most  engineering  disciplines  have  their  foundations  in  applied  mathematics.  Mathematics  is 
then  used  as  a  notation  for  describing  the  artifacts  being  studied  and  as  an  analytical  tool  for 
predicting  the  behavior  of  these  artifacts.  John  Rushby  (SRI  International),  the  first  speaker  in 
the  Formal  Methods  and  Verification  session,  argued  that  mathematical  logic  can  play  this  role 
for  software.  He  explained  that  when  writing  specifications,  one  is  concerned  with  giving  as¬ 
sumptions  about  the  world  in  which  a  system  will  operate,  stating  requirements  concerning 
what  the  system  is  suppose  to  do,  and  describing  how  the  system  will  satisfy  its  requirements. 
Logic  can  oe  used  in  all  three  of  these  tasks.  Moreover,  given  formal  specifications  for  these 
aspects  of  a  system,  it  becomes  possible  to  check  for  various  forms  of  consistency  and  com¬ 
pleteness.  better  understand  the  system  by  posing  challenges,  and  prove  that  a  design  will 
satisfy  the  requirements  under  the  given  assumptions. 

According  to  Rushby,  applying  formal  methods  need  not  be  synonymous  with  performing  com¬ 
plete  proofs  of  correctness.  Researchers  in  formal  methods  are  standardizing  on  four  levels 
of  rigor: 

0.  No  use  of  formal  methods. 

1 .  Using  ideas  from  formal  methods,  but  with  an  ad  hoc  notation  and  proofs 
based  on  ad  hoc  arguments.  This  is  the  way  n  ost  mathematics  is  done  today. 

2.  Using  formal  logic  for  specifications  but  doing  proofs  informally. 

3.  Using  formal  logic  for  specifications  and  employing  automated  theorem 
provers  to  check  proofs. 

One  must  select  the  correct  level  of  rigor  for  the  task  at  hand.  Workers  on  the  two  sides  of  the 
Atlantic  also  seem  to  emphasize  different  ends  of  the  spectrum.  European  work  is  more  con¬ 
cerned  with  levels  1  and  2;  work  in  the  U.S.  has  concentrated  on  level  3. 

Rushby  was  careful  to  discuss  limitations  of  formal  methods.  One  cannot  entirely  formalize  a 
proof,  because  it  is  not  possible  to  verify  the  correctness  of  a  specification  for  requirements  or 
the  accuracy  of  any  formal  description  for  the  physical  reality  that  underlies  a  computing  de¬ 
vice.  Second,  level  3  formal  methods  are  expensive,  much  too  expensive  to  be  used  on  a  sys- 
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tem  of  significant  size.  Gtill,  formal  specifications  do  add  value  during  system  design.  For 
documentation,  they  provide  precise,  unambiguous  descriptions  of  interfaces  and  functions. 
This  in  turn  supports  reuse  of  components.  Having  formal  specifications  also  provides  trace- 
ability  for  design  decisions.  For  analysis,  the  existence  of  formal  specifications  permits  early 
validation  of  requirements  and  helps  to  identify  assumptions  that  must  be  validated  through 
other  means.  Finally,  formal  methods  are  the  only  way  to  verify  “negative”  properties  (i.e.,  the 
absence  of  unintended  effects). 

In  reflecting  on  the  current  state  of  the  art,  Rushby  identified  three  traditions  that  now  seem  to 
be  converging.  The  first  tradition  starts  with  a  specification  notation,  where  an  expressive  no¬ 
tation  is  chosen  without  concern  for  how  difficult  it  would  be  to  construct  a  supporting  theorem 
prover.  The  second  tradition  is  motivated  by  a  drive  to  build  powerful  theorem  provers.  The 
logic  of  such  a  theorem  prover  is  then  used  as  the  specification  notation.  A  third  tradition,  now 
becoming  prevalent,  seeks  to  combine  the  other  two  by  making  compromises  in  the  specifica¬ 
tion  language  and  power  of  the  theorem  prover  in  order  to  obtain  a  coherent  system. 

2.2  Application  Lecture:  Dr.  David  L.  Parnas 

David  Pamas  (McMaster  University)  was  the  second  speaker  of  the  session.  Pamas  has  used 
formal  methods  in  two  large  mission-critical  systems:  the  software  that  controls  a  U.S.  fighter 
plane  (the  A7)  and  the  safety-critical  software  for  a  nuclear  power  station  in  Darlington,  Ontar¬ 
io.  Pamas'  talk  outlined  experiences  with  a  formal  specification  notation  that  he  used  in  the 
Darlington  project.  The  notation  is  easily  read  by  engineers,  making  it  possible  for  the  client  to 
understand  specifications.  Parnas  emphasized  the  importance  of  this  attribute,  as  it  allows  the 
customer  to  catch  errors  that  software  engineers  are  iiot  qualified  to  find.  And  although  the  no¬ 
tation  looks  informal,  it  has  precise  Predicate  Logic  semantics.  Thus,  the  specification  notation 
enjoys  the  same  benefits  and  potential  for  automated  analysis  as  any  specification  notation 
based  on  a  formal  logic. 

According  to  Pamas,  success  in  implementing  safety-critical  software  depends  on  three  ele¬ 
ments.  First,  one  must  be  prepared  to  write  precise,  organized,  mathematical  documentation 
and  review  it  systematically.  Second,  extensive  testing  is  necessary  to  allow  quick  discovery 
of  gross  errors  and  shared  oversights.  Third  (and  perhaps  the  most  worrisome)  is  that  qualified 
people  must  be  involved  and  an  approved  process  employed.  Pamas  has  found  that  writing 
and  reviewing  rigorous  specifications  is  basically  a  boring  task.  Without  people  having  the  right 
skills,  it  is  simply  too  easy  to  overlook  a  flaw.  In  the  Darlington  project,  non-mathematicians 
were  employed  in  writing  and  analyzing  the  specifications.  All  the  participants  required  exten¬ 
sive  training  and  supervision  in  the  formal  methods  being  used  by  the  project.  Being  conver¬ 
sant  in  the  application,  however,  meant  that  the  participants  were  effective  in  reviewing 
specifications. 

The  Darlington  project  and  the  A7  project  both  employed  level  2  formal  methods.  Proofs  re¬ 
mained  informal,  and  only  a  limited  degree  of  automated  support  was  available.  Parnas  ar¬ 
gued  that  formal  specifications  should  be  used  for  documentation  and  specification  now. 


6 


CMU/SE I-93-SR-4 


Otherwise,  we  can  have  no  hope  of  using  them  in  the  future  as  input  to  rigorous  mathematical 
analysis.  The  success  of  the  Darlington  project  certainly  argues  that  level  2  formal  methods 
can  have  a  real  payoff. 

2.3  Panel  Discussion 

The  next  four  presentations  were  from  people  who  had  used  or  tried  to  use  formal  methods. 
Ricky  Butler  (NASA  Langley  Research  Center)  was  the  first  speaker.  Butler  is  trying  to  proto¬ 
type  a  fully  verified  reliable  computing  platform  for  use  in  flight  control.  His  group  has  specified 
and  verified  redundancy  management,  task  scheduling,  and  clock  synchronization  protocols 
for  such  a  system.  They  have  had  experience  using  many  of  the  currently  available  tools — 
Butler  reports  that  it  takes  about  6  months  for  him  to  master  a  new  theorem  prover.  Still,  in 
contrast  to  Pamas,  Butler  believes  that  wrestling  with  a  mechanical  theorem  prover  is  a  nec¬ 
essary  prerequisite  for  writing  good  specifications.  Butler  has  found,  however,  that  the  techni¬ 
calities  associated  with  getting  a  proof  through  a  mechanical  theorem  prover  are  a  distraction. 

Jon  Dehn  (IBM  Federal  Systems  Company)  next  spoke  about  his  experiences  in  using  formal 
methods  for  the  development  of  AAS,  the  next-generation  Air  Traffic  Control  System  that  he 
is  working  on.  This  is  a  large  system — over  1  million  lines  of  source  code  (Ada),  with  each  in¬ 
stallation  running  on  a  network  of  over  200  processors  linked  by  several  local  area  networks. 
The  system  must  satisfy  stringent  reliability  requirements.  Dehn  reported  that  his  successes 
with  formal  methods  were  all  achieved  when  small  teams  had  to  solve  particular  aspects  of  a 
problem  or  when  the  method  being  employed  could  be  mechanized  and  applied  to  an  entire 
system  or  subset,  as  a  success,  Dehn  cited  the  technique  being  used  to  manage  the  applica¬ 
tion  state  data.  Here,  formal  analysis  allowed  potential  race-conditions  to  be  detected  auto¬ 
matically.  Dehn  cautioned  that  formal  methods  have  not  worked  when  appropriate 
mechanizations  did  not  exist  and  when  the  method  jeing  employed  involved  too  many  ques¬ 
tionable  assumptions  about  the  system  being  analyzed. 

The  third  presentation  was  by  Bill  McKeeman  (DEC)  about  his  experiences  moving  the 
LARCH  specification  language  and  method  from  Digital’s  research  laboratory  into  an  industrial 
software  development  organization.  The  plan  was  to  extend  LARCH  to  support  the  C  program¬ 
ming  language  and  then  use  LARCH  to  support  commercial  development  activities.  Largely 
for  cultural  reasons,  the  desired  technology  transfer  did  not  occur.  The  impediments  to  adopt 
ing  the  system  included  the  following.  First,  a  significant  fraction  of  the  user  community  were 
excluded  from  participating  by  the  choice  of  C  as  the  language  being  supported.  Second,  us¬ 
ing  a  system  like  LARCH  is  really  feasible  only  for  new  code,  and  most  software  work  by  the 
target  users  was  concerned  with  extending  existing  code.  And  finally,  the  engineers  regarded 
formal  specifications  as  constraining  their  designs  and  did  not  perceive  a  real  pay  off  in  the 
time  horizon  of  their  projects. 

The  final  presentation  was  by  Fred  Schneider  (Cornell).  Schneider  argued  that  formal  meth¬ 
ods  could  have  leverage  today  in  analyzing  small  parts  of  systems  and  in  analyzing  simple 
properties  of  entire  systems.  Based  on  his  experience  in  applying  formal  methods  at  IBM  and 
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DEC,  he  proposed  that  the  position  of  a  “format  methodist”  be  added  to  the  skills  roster  of  sys¬ 
tems  projects.  Such  a  person  would  design  special-purpose,  lightweight  and  disposable  for¬ 
mal  methods  for  the  problem  at  hand.  Being  successful,  however,  would  require  that  a  formal 
methodist  be  an  active  participant  in  the  system  design — to  understand  what  are  the  real  prob¬ 
lems  and  what  approximations  of  reality  are  possible.  It  also  would  require  access  to  powerful 
and  flexible  tools  that  can  be  easily  integrated. 
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3  Requirements 


The  afternoon  session  of  Thursday,  March  18, 1993  focused  on  the  problem  of  specifying  re¬ 
quirements  for  dependable  software  systems.  The  technology  lecture  was  given  by  Prof.  John 
A.  McDermid  from  the  University  of  York.  The  title  of  his  talk  was  "Dependability  Require¬ 
ments:  Orthodoxy  and  a  Goal-Structured  Approach.”  Mr.  Gerry  E.  Pasek  of  Lockheed  Missile 
and  Space  gave  the  application  lecture,  "Dependable  Software:  A  Challenge.”  In  addition  to 
these  speakers,  the  panel  consisted  of  Dr.  Walter  L  Heimerdinger  of  Honeywell  Systems  and 
Research  Center,  Dr.  Cart  Landwehr  of  the  Naval  Research  Laboratory,  Dr.  David  Pamas  of 
McMaster  University,  and  Dr.  John  Rushby  of  SRI  International.  The  requirements  session 
was  coordinated  by  Dr.  Heimerdinger. 

3.1  Application  Lecture:  Prof.  John  A.  McDermid 

Requirements  analysis  is  a  difficult  subject,  made  all  the  more  so  because  the  customer  may 
think  that  the  requirement  means  one  thing  while  the  developer  sees  it  as  something  altogeth¬ 
er  different.  The  first  speaker,  John  McDermid  (University  of  York),  claimed  that  in  an  attempt 
to  overcome  this,  requirements  tend  to  be  overly  detailed,  bordering  on  implementations.  This 
gives  the  system  designers  and  implemented  little  freedom,  and  hampers  the  development  of 
efficient  systems. 

According  to  McDermid,  the  usual  requirements  specification  is  not  much  more  than  a  list  of 
function  definitions  that  cover  normal  functioning,  abnormal  functioning,  dependability  proper¬ 
ties,  performance,  quality,  and  expected  changes.  These  specifications  contain  too  much  de¬ 
tail.  forcing  early  design  decisions.  This  hampers  flexibility,  making  it  difficult  to  incorporate 
down-stream  changes.  Furthermore,  it  is  hard  to  check  the  consistency  between  requirements 
so  specified. 

What  is  needed,  McDermid  argued,  is  a  systematic  basis  for  the  specification  of  the  fundamen¬ 
tal  requirements  of  the  system.  These  are  the  top-level  goals  of  the  stakeholders  (e.g.,  the  cus¬ 
tomer,  the  users,  etc.).  The  fundamental  requirements  are  then  refined  into  a  set  of  derived 
requirements  that  take  into  account  constraints  of  the  real-world  (e.g.,  size  limitations,  power 
consumption,  etc.).  McDermid  suggests  that,  in  general,  there  is  still  a  need  to  identify  conflicts 
between  requirements  and  to  devise  strategies  that  provide  acceptable  trade-offs.  Depend¬ 
ability  requirements  often  expose  conflicts. 

Representing  goals  precisely  is  a  difficult  problem,  according  to  McDermid.  All  of  the  assump¬ 
tions  behind  the  goals  must  be  enumerated  systematically,  even  hidden  assumptions  (e.g., 
the  aerodynamics  of  each  wing  of  an  airplane  are  different).  Constraints  must  also  be  enumer¬ 
ated,  and  each  goal  has  to  have  an  "owning”  stakeholder  so  that  trade-offs  can  be  evaluated, 
perhaps  by  negotiating  between  stakeholders. 
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In  addition  to  goals,  McDermid  asserts  that  there  are  three  other  components  to  a  requirement: 
there  must  be  a  way  to  teil  when  a  requirement  has  been  met,  there  must  be  a  justification  for 
the  requirement,  and  finally  there  must  be  a  strategy  for  achieving  the  requirement  (e.g.,  goal 
sub-structuring). 

A  conceptual  process  for  requirements  definition,  then,  is  to: 

•  Identify  goals — the  stakeholders'  main  aims  and  desires. 

•  Model  the  environment  and  causal  properties. 

•  Derive  more  detailed  requirements  based  on  real-world  constraints. 

McDermid  was  not  aware  of  any  single  tool  that  can  take  the  designer  through  this  process. 
However,  tools  such  as  STEPS,  ARE,  KAOS  and  TARDIS  can  form  a  base. 

3.2  Application  Lecture:  Mr.  Gerry  E.  Pasek 

The  practitioner’s  viewpoint  was  presented  by  Mr.  Gerry  Pasek  (Lockheed).  He  suggested  that 
dependable  software  development  is  an  iterative  process  that  is  almost  "done  in"  by  require¬ 
ments  analysis.  Just  when  a  system  starts  to  satisfy  the  requirements,  as  specified,  they 
change,  or  the  customer’s  expectations  change.  So  the  current  paradigm  is  to  "code  a  little, 
test  a  little,  and  document  some.”  Pasek  considers  2167A  too  rigid,  and  primarily  for  bureau¬ 
crats  and  lawyers. 

According  to  Pasek,  the  realities  of  the  procurement  process  make  requirements  analysis 
even  more  important  than  it  should  be,  but  at  the  same  time  the  requirements  hobble  innova¬ 
tion.  For  instance,  the  development  of  the  F22  fighter  has  an  integrated  10-year  master  plan 
with  monthly  milestones  to  be  met.  This  means  design  decisions  are  made  entirely  too  early 
in  the  development  process. 

In  the  normal  case,  there  is  little  if  any  user  involvement  in  the  requirements  specification  pro¬ 
cess.  Since  developers  and  users  have  different  viewpoints,  this  is  detrimental  to  the  quality 
of  the  final  system.  Even  in  testing,  developers  and  users  will  base  their  testing  on  their  indi¬ 
vidual  viewpoints  of  what  the  system  should  do.  Pasek  gave  the  example  of  an  F22  crash  at 
Edwards  AFB  that  was  traced  to  a  pilot  misuse  of  the  aircraft,  which  the  software  wasn’t  de¬ 
signed  to  handle.  The  software  failed,  and  its  error  recovery  mechanism  was  faulty.  Too  much 
attention  is  given  to  testing,  and  not  enough  to  specifying,  documenting,  and  validating  user 
functionality. 

Another  problem,  Pasek  claimed,  is  the  incredible  number  of  requirements  typical  for  a  large 
system.  As  the  number  of  requirements  increases,  it  gets  difficult  to  deal  with  interactions.  In 
one  system,  there  were  300  system-level  requirements,  40  real  requirements,  and  45,000 
specific  requirements.  Until  recently,  it  wasn't  possible  to  trace  the  requirements  chain  from 
the  specific  to  the  general.  In  addition  to  tracability  back  to  the  stakeholder/owner  of  a  require- 
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ment,  the  rationale  behind  each  requirement  must  be  documented.  This  allows  the  need  for  a 
specific  requirement  to  be  challenged  and  defended.  Electronic  communications  is  helping 
here,  and  database  tools  are  being  developed,  but  tracability  remains  a  major  problem. 

From  a  dependability  standpoint,  too  little  attention  is  given  to  defining  what  can  go  wrong. 
Systems  are  therefore  inherently  undependable  in  their  initial  phases.  Pasek  suggested  that 
quality  does  not  imply  dependability.  Specifying  the  fault  set  (especially  credible  faults)  to  be 
accommodated  is  a  major  challenge,  especially  if  the  hardware  has  not  been  defined. 

Pasek  was  a  big  fan  of  the  SEI  level-three  process,  which  he  claimed  goes  a  long  way  toward 
getting  the  “right  stuff  into  products,  but  he  feels  we  need  to  concentrate  more  on  defining  the 
project  before  it  starts.  In  particular,  there  needs  to  be  a  way  to  capture  the  thought  process 
leading  to  requirements. 

3.3  Panel  Discussion 

Walt  Heimerdinger  (Honeywell)  started  the  panel  discussion  by  talking  about  requirements 
that  have  worked  well  and  those  that  have  not.  He  divides  the  former  into  two  classes:  process 
and  function.  By  “process,”  he  means  how  the  product  is  built.  This  covers  design  constraints, 
reviews,  etc.  As  examples,  he  cited  DO-1 78A  and  MOD  55  (U.K.  Ministry  of  Defense).  They 
are  relatively  easy  to  specify  and  follow,  but  they  may  not  correlate  with  the  actual  behavior  of 
the  product.  By  “function,”  he  means  what  the  product  does.  This  includes  state  machines, 
scenarios,  fault-tree  analysis,  etc.  These  can  help  to  identify  specific  faults  and  hazards  and 
can  include  specific  error-handing  procedures. 

According  to  Heimerdinger,  requirements  that  don’t  work  well  include  attributes  (because  it  is 
difficult  to  cover  the  entire  product),  and  statistical  objectives  (because  it  is  hard  to  get  a  sta¬ 
tistically  significant  sample  for  ultradependable  systems).  Heimerdinger  suggests  reuse  and 
benchmarks  as  possible  solutions  to  these  problems. 

Carl  Landwehr  (NRL)  talked  about  domains  where  he  has  dealt  with  requirements  for  depend¬ 
able  software.  These  included  security,  safety,  and  real-time  systems.  Techniques  that  have 
served  him  well  included  natural  language,  cookbooks  (e.g.,  the  Yellow  Book,  MOD  55/56), 
structured  informal  techniques  (e.g.,  assertions),  semi-formal  techniques  (e.g.,  SCR),  and  for¬ 
mal  techniques,  both  manual  and  automated.  The  techniques  have  proven  useful  for  capturing 
intended  critical  properties  precisely,  facilitating  communications  between  stakeholders,  re¬ 
vealing  conflicts  and  errors  early  in  system  development,  easing  the  design  process,  facilitat¬ 
ing  system  assurance,  and  training  users.  On  the  flip  side,  available  techniques  don’t  work  well 
when  dealing  with  combinations  of  critical  requirements  or  in  supporting  the  composition/de¬ 
composition  of  requirements  onto  design  entities,  especially  in  the  realm  of  security. 

David  Pamas  (McMaster  University)  contested  at  least  one  of  Heimerdinger’s  points,  stating 
his  belief  that  some  of  the  process  mechanisms  hinder  the  precise  specification  of  require¬ 
ments.  Many  aspects  of  a  system  are  never  documented  because  they  never  cause  trouble. 


CMU/SEI-93-SR-4 


11 


The  final  panelist  was  John  Rushby  (SRI  International),  who  discussed  areas  where  SRI  In¬ 
ternational  had  specified  dependability  properties.  These  included  fault-tolerance  for  flight 
control,  the  “Jet  Select"  function  for  the  Space  Shuttle  Orbit  DAP,  Microprogram  correctness, 
simple  real-time  properties,  and  security  properties.  In  general  this  worked  well.  Expressive 
formal  methods  can  be  used  to  specify  almost  anything  in  an  easily  understood  way,  given  tal¬ 
ented  and  skilled  users.  The  crucial  need  is  for  transferable  methodologies  tailored  to  each 
application  area.  In  almost  ail  areas,  considerable  work  is  needed  to  create  transferable  meth¬ 
odologies. 

Comments  raised  by  members  of  the  audience,  led  by  Herb  Hecht  and  George  Gilley,  involved 
the  procurement  process.  The  procurement  process  is  the  major  problem.  You  are  serving  an 
amorphous,  volatile,  customer  due  to  frequent  personnel  changes.  Hundreds  of  people  are  in¬ 
volved,  each  with  a  different  set  of  expectations.  The  result  is  a  changing  set  of  requirements 
every  time  a  new  person  is  put  in  charge.  There  is  only  one  shot  at  getting  the  requirements 
right,  even  though  iteration  will  be  inevitable.  If  a  requirement  is  eliminated  downstream,  some 
bean-counter  is  going  to  want  a  “give-back”. 
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4  Operating  System  Support 


The  morning  session  of  Friday,  March  19, 1993,  focused  on  operating  system  support  for  de¬ 
pendable  software.  The  technology  lecture  was  given  by  Dr.  Keith  Marzullo  of  Cornell  and  the 
University  of  California  at  San  Diego.  The  title  of  his  talk  was  “Distributed  Fault-Tolerant  Pro¬ 
gramming  Using  Group  Programming  Tools."  Dr.  Doug  Jewett  of  Tandem  Computers  gave  the 
application  lecture,  “Tandem  Fault  Tolerant  Systems."  In  addition  to  the  speakers,  the  panel 
consisted  of  Dr.  Edward  Balkovich  of  Digital  Equipment,  Mr.  Jon  Dehn,  of  IBM  Federal  Sys¬ 
tems,  Mr.  Craig  Hatfield  of  IBM  Federal  Systems,  and  Dr.  Alfred  Spector  of  Transarc.  This  top¬ 
ic  area  was  coordinated  by  Dr.  Ravi  Iyer  of  the  University  of  Illinois  and  the  session  was 
chaired  by  Dr.  Charles  B.  Weinstock  of  the  Software  Engineering  Institute. 

4.1  Technology  Lecture:  Dr.  Keith  Marzullo 

Abstractions  for  implementing  fault  tolerance  are  frequently  supported  in  an  operating  system. 
Consequently,  this  session  concerned  the  various  abstractions  that  have  been  proposed  for 
supporting  fault-tolerance  and  included  representative  users’  experiences  with  these  abstrac¬ 
tions.  The  first  speaker  was  Keith  Marzullo  (Comell/U.C.  San  Diego).  Marzullo  summarized 
the  “group  programming”  approach  to  distributed  computing.  This  approach,  typified  by  the 
ISIS  system,  allows  the  programmer  to  assume  that  events  are  seen  in  the  same  order  by  all 
members  of  a  distributed  group  of  processes.  Having  the  illusion  of  a  total  order  on  distributed 
events  simplifies  programming.  Solutions  to  the  election  problem  illustrated  simplifications  that 
become  possible  when  primitives  to  support  group  programming  are  available. 

Marzullo  explained  that  systems  to  support  group  programming  usually  implement  certain  key 
abstractions.  At  the  lower  levels  are  protocols  to  implement  the  causal  delivery  of  a  message 
to  the  members  of  a  process  group,  protocols  to  detect  failures  of  members,  and  protocols  to 
maintain  a  consistent  view  of  a  group's  membership.  High-level  abstractions  found  usually  in¬ 
clude  ordered  multicasts,  state  transfer  protocols  so  that  new  members  can  be  added  to  a 
group,  coordinator-cohort  and  other  group  coordination  protocols,  and  support  for  logging  and 
recovery. 

As  a  non-trivial  example  of  group  programming,  Marzullo  described  his  RNFS  replicated  NFS 
file  system.  This  system  provides  to  clients  the  same  interface  as  the  SUN  NFS  network  file 
system.  However,  in  RNFS,  files  can  be  replicated  on  multiple  file  servers.  Different  replicas 
are  kept  consistent  by  RNFS.  Marzullo  also  described  ongoing  projects  that  are  exploring 
group  programming  support.  Although  the  list  is  dominated  by  university  research  efforts,  the 
approach  has  been  employed,  for  example,  by  IBM  in  two  product  efforts  (AAS  air  traffic  con¬ 
trol  and  TSAF  VM/370  support). 
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4.2  Application  Lecture:  Dr.  Doug  Jewett 

Doug  Jewett  (Tandem  Computers  Inc.),  the  second  speaker  in  this  session,  surveyed  Tan¬ 
dem’s  efforts  to  provide  fault-tolerant  computing.  Their  first  and  best  known  offering  was  built 
in  the  mid  1970's  using  a  custom  operating  system  running  on  top  of  custom  hardware.  The 
target  customer  application  was  online  transaction  processing.  Tandem’s  system  is  based  on 
using  “process-pairs”  for  system  processes.  Each  process  pair  has  a  primary  and  backup.  The 
primaries  write  periodic  state  checkpoints,  which  allows  a  backup  process  to  take  over  when 
a  primary  process  fails.  Unfortunately,  Tandem  has  found  that  designing  process-pairs  is  dif¬ 
ficult.  Moreover,  the  mechanism  is  not  immune  to  correlated  failures  arising  from  programmer 
errors. 

The  difficulty  in  building  process-pairs  led  Tandem  to  implement  operating  system  support  for 
transactions.  Again,  a  primary  and  backup  are  employed,  but  now  the  usual  transaction  se¬ 
mantics  simplifies  the  checkpointing  problem.  By  writing  transactions  rather  than  process- 
pairs,  users  of  Tandem  systems  are  able  to  implement  fault-tolerance  in  their  applications.  The 
current  system,  then,  is  a  hybrid:  critical  software  (including  a  transaction  manager)  is  written 
as  process-pairs  and  gives  continuous  service  across  failures;  most  user  software  is  written  in 
terms  of  transactions  and,  therefore,  is  restarted  in  response  to  a  failure. 

Tandem  is  also  designing  a  fault-tolerant  UNIX  system.  During  his  lecture.  Jewett  briefly  out¬ 
lined  this  system  as  well.  The  goal  is  to  support  existing  UNIX  applications  program  interfaces 
and  be  able  to  survive  hardware  failures.  A  variety  of  design  changes  are  being  used  to  harden 
the  kernel,  as  UNIX  was  not  originally  intended  for  this  domain.  Jewett  also  discussed  some 
of  the  difficulties  encountered  in  debugging  such  a  system. 

4.3  Panel  Discussion 

The  next  four  presentations  discussed  systems  efforts  to  support  fault-tolerance.  Edward 
Balkovich  (DEC)  started  by  describing  three  representative  system  architectures  for  imple¬ 
menting  different  degrees  of  fault-tolerance.  From  these,  Balkovich  then  concluded  that  man¬ 
aging  redundancy  in  storage,  processing,  and  the  communications  network  must  be  integral 
to  the  design  of  an  operating  system.  The  use  of  journaling  (i.e.,  logging  on  stable  storage) 
was  found  to  be  quite  useful  for  implementing  critical  services  and  is  frequently  used  in  DEC 
systems  applications.  Balkovich  also  pointed  out  that  fault-tolerance  support  could  be  em¬ 
ployed  to  allow  system  upgrades  without  shutting  down  a  system. 

The  next  presentation  was  by  Craig  Hatfield  (IBM  Federal  Systems  Company)  concerning  an 
operating  system  that  IBM  is  building  for  a  defense  application.  Among  the  system’s  capabil¬ 
ities  are  replication  of  critical  files  and  restart  without  reloading.  The  system  runs  on  multiple 
communicating  processors  and  has  a  master  and  slaves.  A  slave  can  take  over  for  the  master, 
if  that  master  fails.  Echoing  Tandem’s  experiences,  Hatfield  found  that  writing  applications 
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programs  was  complicated  by  the  need  to  take  checkpoints  for  failure  recovery.  Hatfield  also 
reported  that  coping  with  power  failures  was  particularly  difficult  because  some  hardware  in¬ 
terfaces  had  to  be  reinitialized  following  power  outages. 

Alfred  Spector  (Transarc)  then  reported  on  the  use  of  transactions  as  a  way  of  providing  fault- 
tolerance.  Spector  discussed  elements  of  current  OSF  DCE  and  Transarc’s  Encina  systems 
in  order  to  illustrate  the  practicality  and  acceptance  of  this  approach.  The  use  and  implemen¬ 
tation  of  transactions  is  well  understood;  the  hardest  part  at  this  point  is  getting  all  the  details 
right  so  that  an  implementation  correctly  deals  with  system  partitions  and  heterogeneous  pro¬ 
cessors.  Spector  did  identify  some  outstanding  research  issues.  Work  needs  to  be  done  in  un¬ 
derstanding  replicated  datatypes  and  in  automated  support  for  application  development. 
Spector  also  called  for  simplified  administration  of  replicated  systems. 

The  final  speaker  of  the  session  was  Jon  Oehn  (IBM  Federal  Systems  Company).  Dehn  briefly 
outlined  how  group  programming  is  employed  in  IBM's  AAS  system.  Application  programmers 
do  not  see  a  group  programming  interface.  Rather,  group  programming  is  used  to  implement 
a  primary-backup  style  of  fault-tolerance.  The  replication  that  group  programming  would  entail 
for  the  application  levels  of  the  system  was  deemed  to  be  too  expensive  to  be  practical. 
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5  Object-Oriented  Programming  and  Design 


The  afternoon  session  of  Friday,  March  19,  1993,  focused  on  object-oriented  programming 
and  design  as  a  means  of  achieving  dependable  software.  The  technology  lecture  was  given 
by  Dr.  Gul  Agha  of  the  University  of  Illinois.  The  title  of  his  lecture  was  “Object-Oriented  De¬ 
pendable  Computing.”  Dr.  T.  L.  Wang  of  AT&T  Bell  Laboratories  gave  the  application  lecture, 
“Object  Oriented  Design  of  a  Highly  Available  Switching  System.”  In  addition  to  these  speak¬ 
ers,  the  panel  consisted  of  Dr.  Jacob  Abraham  of  the  University  of  Texas  at  Austin,  Dr.  Jim 
Coplien  of  AT&T  Bell  Laboratories,  Mr.  Bran  Selic  of  ObjecTime,  and  Dr.  Peter  Wegner  of 
Brown  University.  The  object-oriented  programming  and  design  topic  was  coordinated  by  Dr. 
Yitzak  Levendel  of  AT&T  Bell  Laboratories  and  Dr.  Gul  Agha  of  the  University  of  Illinois. 

5.1  Technology  Lecture:  Dr.  Gul  Agha 

Dr.  Agha’s  (University  of  Illinois)  talk  began  with  a  short  tutorial  on  object-oriented  program¬ 
ming.  He  discussed  procedure  abstraction,  data  encapsulation,  data  abstraction,  and  the  idea 
of  object  inheritance.  The  key  use  of  abstractions  is  to  separate  how  the  code  is  implemented 
from  its  functional  specification.  It  is  also  common  to  want  to  separate  the  time  that  something 
is  done  from  the  actual  task.  For  instance,  asynchronous  communication  allows  computation 
to  proceed  in  parallel  with  that  communication.  This  leads  to  the  need  for  synchronization. 

According  to  Agha,  the  advantage  of  object  orientation  are  in  data  abstraction,  modeling  pow¬ 
er,  and  reuse.  Abstractions  are  provided  by  encapsulation  and  hidden  representation.  Data 
abstraction’s  modeling  power  comes  from  the  hierarchical  organization  and  through  special¬ 
ization  (achieved  through  inheritance).  Reuse  is  achieved  through  incremental  refinement  of 
software  components. 

Agha  then  went  on  to  describe  a  methodology  for  dependability.  The  four  pieces  of  this  meth¬ 
odology  include  modularity,  reusability,  flexibility,  and  composability.  He  introduced  the  actor 
model.  In  this  model,  the  universe  contains  computational  agents  called  actors.  Each  actor  has 
a  mail  address,  a  behavior,  and  a  local  state.  Behaviors  can  be  dynamically  replaced  and  new 
actors  can  be  created.  Actors  communicate  using  a  point-to-point,  asynchronous,  buffered 
protocol.  Mail  addresses  can  be  communicated,  which  allows  for  dynamic  topology.  Arrival  or¬ 
der  and  activation  order  form  the  fundamental  synchronization  mechanisms. 

Agha  believes  that  three  aspects  of  the  behavior  of  an  actor  are  necessary  and  sufficient  for 
most  dependability  protocols.  These  are  the  structure  of  its  mail  queue,  the  dispatcher  that 
handles  message  sends,  and  the  state  of  the  actor.  Each  of  these  components  is  a  separate 
object,  independent  of  the  application.  He  then  went  on  to  describe  the  Broadway  Kernel  and 
the  HAL  language. 

As  an  example  of  a  specific  dependability  mechanism,  Agha  showed  how  actors  can  be  rep¬ 
licated.  The  changes  are  relatively  minor.  The  dispatcher  in  new  copies  of  the  actor  is  modified 
to  send  all  messages  to  the  original  actor’s  dispatcher.  The  original  actor’s  mail  queue  is  mod- 
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ified  to  broadcast  to  all  of  the  replicated  copies,  and  the  behavior  of  the  original  actor’s  dis¬ 
patcher  is  modified  to  collect  messages  from  the  copies,  vote  the  results,  and  pass  the 
resulting  message  to  other  actors. 

5.2  Application  Lecture:  Dr.  T.  L.  Wang 

Dr.  Wang  (AT&T)  discussed  applying  object  oriented  methods  to  the  design  of  a  highly  avail¬ 
able  switching  system.  Switching  systems  are  real-time  systems  where  continuous  operation 
is  of  paramount  importance.  Some  errors  (e.g.,  dropped  or  misrouted  calls)  are  tolerable,  as 
the  customer  can  retry  or  redial.  But  once  a  call  is  completed,  the  goal  is  that  the  call  be  main¬ 
tained.  The  switch  has  a  three  minutes  per  year  maximum  downtime  requirement.  Over  time, 
they  have  observed  that  outages  are  caused  by  hardware  failures  20%  of  the  time,  software 
failures  40%  of  the  time,  and  procedure  failures  40%  of  the  time.  The  design  goal  of  high  avail¬ 
ability  means  that  repairs  must  be  made  quickly  when  there  is  a  failure.  A  major  research  issue 
is  software  repair. 

As  Wang  described  it,  the  logical  hardware  architecture  of  the  switch  is  functionally  distributed 
across  several  processor  families.  These  include  call  processing,  operation,  accounting  and 
maintenance,  interconnect,  and  peripheral  control.  Each  of  these  families  is  its  own  environ¬ 
ment  with  its  own  availability  strategy.  Fault  tolerance  is  provided  by  having  spares  for  each 
family.  They  are  considering  a  heterogeneous  environment  (different  architectures)  to  avoid 
common  mode  failures. 

The  software  architecture  is  component  oriented.  All  of  the  components  are  based  on  the 
same  basic  framework.  Implementations  of  specific  components  use  inheritance  to  help  im¬ 
plement  the  specific  functionality.  The  architecture  supports  flexible  distribution  of  software 
components.  Wang  suggested  that  this  allows  for  load  balancing  and  for  dynamically  relocat¬ 
ing  software  modules  to  minimize  the  size  of  repair  groups.  The  architecture  is  made  of  both 
static  components  (e.g.,  those  representing  the  switch  itself)  and  dynamic  objects  (e.g.,  those 
representing  a  particular  call). 

The  design  makes  heavy  use  of  an  object  model  with  complete  data  and  text  encapsulation. 
No  single  component  knows  what  any  other  component  is  doing.  Multiple  inheritance  provides 
away  to  resolve  name  conflicts  and  priorities.  Individual  designers  use  incremental  redefinition 
of  classes  and  methods.  According  to  Wang,  this  provides  a  modular  structure  for  the  system 
architecture,  provides  for  localization  of  recovery  and  repair  actions,  and  coordinates  hard¬ 
ware  and  software  initializations.  A  virtual  machine  memory  model  provides  safe  access  to 
program  and  data  objects. 

The  error  checking  aspects  of  the  system  described  by  Wang  include  run-time  type  checking, 
application  signaled  errors,  and  customized  stack  recovery  actions.  This  simplifies  exception 
handler  design  and  recovery  actions,  and  reduces  the  need  for  audits  during  execution. 
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Wang  closed  with  a  summary  of  the  key  attributes  of  the  system  that  enhance  availability, 
which  include  repairable  interfaces,  flexible  software  configuration,  update  on-the-fly,  elimina¬ 
tion  of  memory  errors,  graceful  error  recovery,  run-time  error  detection,  and  standardized 
maintenance  frameworks. 

5.3  Panel  Discussion 

The  panel  discussion  started  with  a  brief  talk  by  Jacob  Abraham  (University  of  Texas).  He  has 
used  object-based  techniques  in  application  software  development  and  as  a  basis  for  increas¬ 
ing  system  dependability.  In  particular,  it  was  used  for  development  of  CAD  tools  for  VLSI  test 
and  verification  as  well  as  for  tools  to  design  and  evaluate  fault-tolerant  systems.  This  includes 
a  fault  injection  system  written  in  C++. 

Object-oriented  techniques  have,  in  general,  worked  well  for  Abraham.  During  the  develop¬ 
ment  of  prototypes,  the  ability  to  change  algorithms,  modify  data  structures,  etc.,  in  parts  of 
the  tool  has  allowed  easy  exploration  of  alternatives.  It  has  also  made  it  easy  to  integrate  tools 
written  by  different  programmers.  However,  there  is  a  need  for  an  object-based  language  that 
can  capture  specifications  of  a  program  or  design.  Also  needed  is  support  for  designing  sys¬ 
tems  seamlessly  at  the  software  and  hardware  levels. 

James  Coplien  (AT&T)  talked  about  “Objects,  Multiple  Paradigm  Integration  and  Organization 
Structure.”  His  thesis  is  that  complexity  is  the  root  of  many  software  woes  and  that  abstraction 
is  one  way  to  attack  complexity.  Object-oriented  techniques  are  one  tool  for  dealing  with  ab¬ 
stractions,  but  they  aren’t  suitable  for  every  domain.  A  good  abstraction  encapsulates  change. 
Object  partitioning  doesn’t  always  accomplish  this  and  may  lead  to  unsuitable  coupling.  Ob¬ 
jects  are  bad  at  encapsulating  change  in  real-time  embedded  systems. 

Bran  Selic  (ObjecTime)  talked  about  work  he  has  done  in  the  telecommunications  area  for 
Northern  Telecom.  The  system  he  is  involved  with  has  over  20  million  lines  of  code,  making 
individual  methods  all  but  invisible.  In  this  system,  requirements  are  continually  being  added, 
the  level  of  distribution  is  changing,  the  structure  is  changing,  and  it  is  mostly  soft  real-time. 

In  this  environment  the  object  model  has  resulted  in  a  significant  productivity  improvement 
(measured  at  5:1).  This  is  due  primarily  to  a  more  appropriate  programming  paradigm,  reuse, 
and  an  exceptional  development  environment  (Smalltalk-80).  The  object  model  is  inherently 
architectural.  The  basic  programming  model  is  a  network  of  interacting  components,  which  is 
a  natural  model  for  distributed  real-time  systems.  According  to  Selic,  the  built-in  abstractions 
mechanisms  encourage  specification  of  evolvable  systems. 

However,  according  to  Selic,  all  is  not  perfect  with  the  architectural  model.  Object-oriented 
concepts  are  unnecessarily  restricted  to  the  programming  language  level,  and  programming 
languages  are  not  conducive  to  architectural  modeling.  Consequently  one  cannot  exploit  use¬ 
ful  object-oriented  concepts  at  the  architectural  level  where  the  payback  would  be  the  greatest. 
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Peter  Wegner’s  talk  concluded  the  object  oriented  panel.  He  presented  a  list  of  research  is¬ 
sues  for  object-oriented  programming  and  design,  including  how  to  use  Ada  in  this  environ¬ 
ment,  interface-oriented  programming,  type  safety  versus  incremental  class  flexibility, 
languages  for  talking  about  interconnection  and  communication,  reflective  architectures,  con¬ 
straints,  c r  ^currency  and  synchronization  of  objects,  persistence,  and  heterogeneous  objects. 

In  the  ensuing  discussion,  one  of  the  key  themes  was  how  object-oriented  programming  and 
design  applies  to  dependability.  One  response  was  that  it  helps  to  constrain  the  design  by  pro¬ 
viding  rules  for  interactions  between  objects.  However,  Jim  Coplien  claimed  that  fault  toler¬ 
ance  needs  a  much  deeper  understanding  of  the  world  than  the  basic  system  design. 
Recovery  is  not  a  part  of  the  objects,  but  rather  it  is  in  the  backplane  of  the  system.  Dr.  Wang 
then  said  that  the  object  paradigm  is  a  starting  point  not  a  complete  solution;  it's  just  easier  to 
start  with  objects. 
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6  Participants’  Comments 


All  of  the  participants  attending  the  Technology  Exchange  were  asked  for  written  comments 
on  the  meeting.  Over  25%  of  the  attendees  provided  this  feedback.  This  section  summarizes 
those  comments. 

Participants  completed  a  form  that  asked  what  their  expectations  about  the  exchange  were 
and  how  well  these  expectations  were  met.  For  the  most  part,  the  participants’  expectations 
were  met: 

•  The  meeting  allowed  participants  to  find  out  about  potentially  useful  state-of- 
the-art  work  in  dependable  software. 

•  The  discussions  helped  set  an  agenda  of  technology  issues  that  are  ripe  for 
further  exploration. 

•  Participants  had  a  chance  to  meet  new  colleagues  and  compare 
experiences. 

In  general,  participants  found  the  selected  topics  to  be  appropriate  (average  4  out  of  ),  the 
technical  depth  of  talks  was  almost  right  (average  3  out  of  5),  and  the  format  was  useful  (av¬ 
erage  4+  out  of  5). 

When  asked  which  talk  they  liked  most  and  which  they  liked  least,  the  responses  were  mixed. 
Most  of  those  responding  found  the  formal  methods  and  verification  session  to  be  outstanding. 
But,  good  and  bad  things  were  said  about  all  of  the  sessions: 

uGul  Agha's  talk,  although  more  abstract,  was  full  of  excellent  future’ 
practical  implementation  suggestions.  Some  moments  in  the  panel 
discussion  were  very  informative.” 

“Gul  Agha's  presentation  was  not  well  thought  out  and  was  not  appropriate 
for  the  audience.” 

The  formal  methods  and  verification  topics  were  useful — these  are  important 
areas  for  our  company." 

The  formal  methods  section  was  interesting  but  provided  little  that  I  can 
implement  in  a  tight  schedule,  minimum  budget  environment.” 

There  was  some  unhappiness  with  the  industry  lectures.  As  one  respondent  put  it,  The  indus¬ 
try  presenters  were  not  that  helpful — either  their  material  was  not  very  thoughtful  or  it  was  de¬ 
scription  of  large,  untractabie  (sic),  definitely  not  safety-critical  systems.”  Yet,  at  the  same  time, 
the  respondents  felt  that  'the  idea  of  having  different  types  of  representation,  to  get  a  good  mix 
of  practitioners  and  technology  makers”  was  worthwhile. 

Nearly  all  of  the  respondents  would  like  to  see  additional  exchanges  (5-  average  out  of  5).  Top¬ 
ics  that  they  suggested  include: 

•  Metrics 

•  Analysis  of  results  on  real  life  cases. 
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•  Implementation  issues  on  real  hardware. 

•  Design  of  small,  real-time  kernels  for  safety  critical  systems. 

•  Testing  of  safety  critical  systems. 

•  More  on  formal  methods. 

•  More  on  requirements  (but  more  focused). 

•  Reliability. 

Most  participants  would  attend  additional  exchanges  and  would  like  to  see  one  held  within  a 
year.  One  respondent  suggested  the  next  exchange  be  in  4  months.  Six  months  was  the  typ¬ 
ical  time  frame  suggested. 

A  mailing  list  "exploder”  depend-sw@sei.cmu.edu  has  been  created  to  reach  the  people  who 
attended  the  exchange. 
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